Loadstar 226/226.d81/t.neural doc

Text File | 2022-08-26 | 12KB | 383 lines

u
   NEURAL NETWORK ON A COMMODORE 64
            by John Walker
          September 4, 1987


       -----------------------

    When you first start to learn
about computers, it's only natural to
compare how a computer "thinks" with
how people do. As you learn more, you
tend to stop making such comparisons
because you come to realize that far
from the literal, dumb servants
they're often pictured as in movies,
computers are completely different
from people both in how they operate
and the kinds of problems they can
solve.

    But it's still fascinating to
compare a computer to a human brain.
Researchers are beginning to discover
principles which may explain how the
brain works, and while much of this
research is extremely complicated and
requires large, expensive computers,
your Commodore 64 can be programmed in
BASIC to mimic one fundamental part of
brain function, pattern recognition.
In doing this, your computer is not
only doing something the brain does
(and your reaction may be "I didn't
know a computer could do that!"), it's
doing it the brain's way--by
simulating the neurons (brain cells)
with which you think.


COMPUTER STORAGE AND HUMAN MEMORY

    First, let's compare what the word
"memory" means in a computer as
opposed to a human being. A computer
memory is really a huge collection of
pigeonholes into which numbers are
stuffed. If you want to store
something that isn't a number, such a
piece of text, a picture, or a
sequence of musical notes, you first
have to convert it to numbers, then
put those numbers into the pigeonholes
of the computer's memory. Machine
language programming is just
translating the instructions for
solving a problem into numbers the
computer can remember. A computer
stores numbers.


ATTNEAVE'S CAT

    Human memory is richer and much
more complicated. What comes to mind
when you look at the picture in Figure
1? Even though this picture is nothing
more than 63 straight black lines on a
white piece of paper, your brain
immediately recognizes it as a
sleeping cat. You can even imagine how
a real cat would look if seen from
other angles. The human brain seems to
store and recall patterns. These
patterns don't have to be pictures.
You can recall pieces of music from
only a few notes of the melody, think
of words that rhyme with "frog" or
that end with "ple", and quickly name
the American presidents that have the
same names as automobiles.

    It isn't that human memory is
better or worse than computer memory,
but rather that they are entirely
different things. Some computer
scientists prefer the term "storage"
to "memory" because it more accurately
describes what the computer does. If
you walk up to a computer and ask it
"what is the name of the famous bridge
in the same state as Disneyland" you
won't have much success, but most
people will immediately answer, "The
Golden Gate Bridge". On the other
hand, if you try to memorize a list of
20,000 two digit numbers, you probably
won't succeed, yet a Commodore 64 can
do this in less than a second and not
make a single mistake. That computer
and human memory are so different
isn't all that surprising when you
consider how differently constructed
are the brain and a computer.


WHAT BRAINS ARE MADE OF

    The portion of the brain believed
responsible for thought and memory
consists primarily of nerve cells, or
neurons. (See Figure 2) Each neuron
has three parts, dendrites, a cell
body, and an axon. The dendrites
connect to the axons of other neurons.
When these other neurons are
stimulated, the dendrites convey the
signal to the cell body via a synapse
or connection, which either excites or
inhibits the neuron (with a different
strength for each synapse). When the
excitation sufficiently outweighs
inhibition, the neuron "fires". This
sends a signal down its axon which in
turn excites or inhibits other
neurons, and perhaps causes a muscle
to move.


A NEURON NETWORK

    Because neurons primarily connect
to other neurons they form networks of
great complexity. Figure 3 shows two
fields of five neurons each, in which
each neuron connects to every neuron
in the other field. In this simple
case we have 10 neurons with 5
connections (or synapses) each, for a
total of 50 synapses. Now consider the
brain. Researchers believe that the
brain contains between ten and a
hundred billion neurons, each of which
connects to anywhere from a thousand
to a hundred thousand other neurons,
forming at least ten trillion
connections, and probably far more.
Compare this to the read-write memory
of the Commodore 64, which is made up
of about a quarter million
transistors, and remember that each
transistor is only a switch--far
simpler than a neuron.


THE BRAIN SIMULATOR

    Ten billion neurons, ten trillion
connections: does it make sense to
talk about simulating the brain on a
computer? Can we make a computer
recognize patterns the way a brain
does? Remarkably, we can. A simple
program can simulate the behavior of a
network of interconnected neurons. You
can show this program patterns and it
will remember them. Then if you show
it a similar pattern, it will find the
pattern it's learned that is most like
the pattern it's shown. The technical
name for this is an "associative
memory", so called because it recalls
items based on similarity, like the
brain, as opposed to location, like a
computer.

    The Brain Simulator is written in
BASIC for the 64 and 128 (in 64 mode).
You'll see two blank windows on the
screen. Below them is a legend that
explains the action of the function
keys. When you type a letter or
number, the dot pattern for that
symbol appears in the left window.

    Try typing a few letters and
numbers to see how this works. When
you start the program, it doesn't know
any patterns -- so we'll teach it
some. To learn a pattern, place it in
the left window by pressing the key
for the letter or number and then
press F1. The program trains its
simulated neurons to memorize the
pattern (this takes about 30 seconds).
READY reappears on the screen when the
pattern has been learned. Go ahead and
teach the program three
different-looking letters, "A", "T",
and "Z".

    Now let's try recalling a pattern.
Press the "A" key to place an A in the
left window. Now press F3--this
introduces errors in the pattern by
randomly changing about 10% of the
dots in the pattern each time you
press it. After you press F3, you'll
have a pattern that looks something
like an A, but doesn't exactly match
what we taught the program.

    Press F5 to start the recall
process. The pattern is run back and
forth through the neuron network until
it stabilizes on a fixed pattern (an
arrow in the middle of the screen
shows the direction of the transfer).
After the neuron network has "thought"
about the problem for a few cycles,
you'll probably get back the original
A we taught the program (just like the
brain, this process doesn't always
remember the right thing; if the
random changes made the pattern more
like another pattern the program has
learned, that one will be found
instead).

    Try entering T and Z, creating
errors in them by pressing F3 one or
two times, and recalling with F5. Note
how the neuron network almost always
recalls the correct pattern even
though you've given it something quite
different from what it learned. Enter
"I" and try recalling with F5. The
network recognizes this as T because I
looks more like T than A or Z, the
other patterns it has learned. This is
what your brain does when it sees a
pattern of lines and immediately
thinks of a sleeping cat. Many
researchers think the basic process
the brain uses is much the same as the
one used by this program.

    You can make the program forget
everything it has learned by pressing
F4. If you press F6, the program exits
to LOADSTAR. When you leave the
program everything it has learned is
forgotten, but you can save learned
patterns on the disk by pressing F7
and entering a file name. The next
time you run the program you can
reload the program's memory by
pressing F8 and entering the same file
name.


HOW IT WORKS

    The remarkable thing about this
program is that it doesn't "know" it's
recognizing letters and numbers. As
far a